vscode: send only LifecycleReporter.enable on the diagnostics socket - #47372

How the numbers were taken. The process is bun 1.4.3-canary.1+367d939d9 (release, Linux x64). The extension code runs from main and from this branch with a stand-in for the vscode module. A relay on the socket counts the messages. Where a row says node, the extension runs under node v26.3.0. The process answers the extension's requests before it starts to log. A script that ends before that sends nothing, on main too. The script at the end of these notes prints the first three rows.

• One-shot commands cost the same on main, with enableConsole: false only, and with this PR: tsc --version 1,011M to 1,012M instructions, prettier --check 1,172M to 1,178M, a loop over 2,000 functions 898M to 902M. The code discard costs a process that did work before its first turn of the event loop.
• bun binary: 0 bytes. No file under src/ changes.
• The error of a live process arrives with the one request on bun 1.1.38 (the first release that reads the variable), 1.2.0, 1.2.23, 1.3.0, 1.3.14, 1.4.0 and 1.4.3-canary.1.
• The bundle is built by esbuild with the options of packages/bun-vscode/scripts/build.mjs. ws, @vscode/debugadapter and semver are external in both builds, because this checkout does not install them.
• perf and strace are not on the machine used. A small perf_event_open program counted the instructions of the process and its threads. An LD_PRELOAD counter on libc send and write counted the socket writes.

Why compiled code is discarded. initialize() calls configurationDone() for a client that is not VS Code's debug UI (adapter.ts:531), also with enableDebugger: false. That sends Debugger.setBreakpointsActive. JavaScriptCore's Debugger::setBreakpointsActivated calls recompileAllJSFunctions() when the value changes, and the value starts as false. bun handles the request on a turn of its event loop, so the code of a synchronous start is what it discards.

Why logged objects stay alive and logged errors print differently. With the Console domain on, ConsoleMessage::addToFrontend wraps each argument in the console object group and builds a preview of it. Only a clear of the console releases that group, and the extension never sends one. The preview reads an error's stack. JavaScriptCore then drops the error's structured frames (ErrorInstance::materializeErrorInfoIfNeeded), and bun's printer and LifecycleReporter.error take the frames from the text (src/jsc/bindings/ZigException.cpp:632). With the domain off, JavaScriptCore still keeps the last 100 console messages and their arguments. That is the 92 in the table.

The errors that arrive less often. bun does not wait for its debugger thread when it exits (#34218, open PR #36524). An error that is printed just before the exit reaches the extension only if that thread wins a race. The six requests kept the main thread busy a little longer. With Debugger.setBreakpointsActive sent after the one request, 22 of 300 errors arrive, against 15 of 300 in the same session without it. An error thrown at once never arrives, on main too: the process ends before it handles the request.

The test.

• CI runs only test/. test/integration/bun-lambda already loads a packages/ module from there, and vscode: run package.json scripts from the clicked package #43785 puts tests in test/integration/bun-vscode/.
• The first test uses a real bun behind a relay. It pins the request list, the events that bun sends, the printed stack and the place of the diagnostic. With diagnostics.ts from main all four differ: six requests, four Console.messageAdded events, at URL (unknown:1:1) in the stack, and a diagnostic at unknown:1:1.
• The second test is a peer that closes before it answers. When the socket is reset and not closed, the existing Inspector.error listener also prints the socket error, as on main.
• USE_SYSTEM_BUN=1 cannot show the failure, because the test loads the TypeScript from the tree. With diagnostics.ts from main, both tests fail: 10 of 10 runs on release, and on the debug build.
• Release: 20 of 20 pass. Debug ASAN: 20 of 20, the first run from an empty page cache. At a36e20c, after the last changes to the setup of the test: release 8 of 8, debug ASAN 6 of 6.
• The relay and the process are created at module scope, and afterAll closes them on every path. beforeAll waits for the process with a 60 s deadline. With the start inside the test, the first run of a debug build took 3.6 s to more than 5 s, and two of five such runs passed the 5 s that a test has. The body of the test takes 0.2 to 0.4 s on the debug build.
• A setup step that cannot finish fails with its cause, not with a timeout. This covers a process that dies and a socket that closes before the answer. It also covers a relay that cannot listen and an extension that does not start its socket.
• Seven variants of the change each fail a test: a second request (Console.enable, Inspector.enable, Inspector.initialized), no .catch, an empty .catch, enableConsole: false only, and main.
• tsc reports 0 errors for the new files. For packages/bun-vscode it reports 105 errors on main and 105 on this branch, none in diagnostics.ts (with the extension's type packages installed).
• The extension's own tests (packages/bun-vscode/src/features/tests/__tests__) are not run by CI and do not load diagnostics.ts. 9 to 12 of the 191 fail on main and on this branch alike.

Not in this PR.

• adapter.ts:497 (enableConsole ?? true) and debug.ts are unchanged. Debug sessions send the same six requests, and the adapter's Console.messageAdded handler is empty, so nothing reads the stream there either. [VSCode] Debugger does not log to debug console #14244 asks for that output in attach sessions. initialize() also sends its requests without a .catch for those callers.
• BunDiagnosticsManager.dispose() returns a disposable and closes nothing, so the socket stays open when the extension is deactivated. On macOS and Linux its file then stays in the temporary directory. The test calls the returned disposable.
• A LifecycleReporter.error event that makes handleLifecycleError throw is an uncaught exception in the data listener of the socket. It is the same on main, where the listener runs inside super.emit (adapter.ts:377).
• The extension's framer drops an inspector message over 1 MB (node-socket-framer.ts:57). With this PR only an error event can be that large.
• The variable itself has costs in the runtime that no request list changes. It turns the transpiler cache off (src/runtime/jsc_hooks.rs:687): tsc --version takes 1,012M instructions with the variable set and 142M without it. A connected process builds a ConsoleMessage for each console call (src/jsc/bindings/ConsoleObject.cpp:39): that is the 4,901 against 2,578.
• Nothing in the runtime bounds what bun buffers on this socket (connectToUnixServer in src/js/internal/debugger.ts writes with socket.$write). inspector: give each frontend its own queue and state, drop one that reads nothing #36848 bounds the --inspect WebSocket writer only.
• A copy of the extension that is already installed keeps the six requests. The extension ships by a manual run of .github/workflows/vscode-release.yml, and this PR does not change the version.
• A process started by hand with --inspect-wait=<the diagnostics socket> no longer runs its script: the client does not send Inspector.initialized. On main it ran the script and then did not exit. No code in bun or the extension starts a process that way.
• Users met this console stream before, with bun repl in VS Code terminals: bun repl in VSC terminal messed up #15588, Bun REPL in VS Code terminal on Windows outputs error #17567 and REPL unusable within VSCode's integrated terminal #25389. Those ended with a workaround in bunx and the native bun repl. No report names slow output or memory: this was found by measurement.
• Remove dead code from the build scripts, the vscode and debug-adapter packages, the bake overlay, and two WebCore files #40122 removes dead code in other lines of diagnostics.ts.
• Self-review items not taken: the state of the published copies of the extension (it could not be read from the machine used), the console stream of Bun.WebView (another subsystem, not checked), and whether bun.diagnosticsSocket.enabled stays on by default (a maintainer's decision). The self-review also asked for tracking issues for the items in this list. They are listed here and not filed.

// Save at the root of a checkout and run: bun test ./measure-diagnostics-socket.test.ts
// It loads the extension's diagnostics feature with a stand-in for `vscode`, puts a relay on its socket,
// and prints what one connected `bun` costs. Run it on main and on this branch.
import { mock, test } from "bun:test";
import * as net from "node:net";
import { join } from "node:path";

const src = join(import.meta.dir, "packages", "bun-vscode", "src");
class Disposable {
  static from = () => new Disposable();
  dispose() {}
}
mock.module("vscode", () => ({
  window: { createOutputChannel: () => ({ appendLine() {} }) },
  languages: { createDiagnosticCollection: () => ({ clear() {}, set() {} }) },
  workspace: { textDocuments: [], onDidChangeTextDocument: () => new Disposable() },
  Uri: { file: (path: string) => ({ path }) },
  ...Object.fromEntries(["Position", "Range", "Location", "Diagnostic", "DiagnosticRelatedInformation", "MarkdownString"].map(name => [name, class {}])),
  DiagnosticSeverity: { Error: 0 },
  Disposable,
}));
mock.module(join(src, "extension.ts"), () => ({ getConfig: () => true }));
const { registerDiagnosticsSocket } = await import(join(src, "features", "diagnostics", "diagnostics.ts"));
const env: Record<string, string> = {};
await registerDiagnosticsSocket({ subscriptions: [], environmentVariableCollection: { clear() {}, replace: (k: string, v: string) => void (env[k] = v) } });
const extension = env.BUN_INSPECT_CONNECT_TO;

/** Calls `onMessage` for each inspector message of a stream: a 4-byte length, then JSON. */
function messages(onMessage: (message: any, bytes: number) => void) {
  let buffered = Buffer.alloc(0);
  return (chunk: Buffer) => {
    buffered = Buffer.concat([buffered, chunk]);
    while (buffered.length >= 4 && buffered.length >= 4 + buffered.readUInt32BE(0)) {
      const end = 4 + buffered.readUInt32BE(0);
      onMessage(JSON.parse(buffered.toString("utf8", 4, end)), end);
      buffered = buffered.subarray(end);
    }
  };
}

/**
 * Runs a bun that is connected to the extension through a relay. `prelude` runs at once, before bun handles
 * the extension's requests. `script` runs after bun has answered them.
 */
async function run(script: string, prelude = "") {
  const requests: string[] = [];
  const console = { frames: 0, bytes: 0 };
  let last = -1;
  let socket!: net.Socket;
  let answered = Promise.withResolvers<void>();
  const server = net.createServer(bun => {
    socket = bun;
    const upstream = extension.startsWith("unix://") ? net.connect(extension.slice(7)) : net.connect(Number(new URL(extension).port), "127.0.0.1");
    upstream.on("data", messages(({ id, method }) => (requests.push(method), (last = id))));
    bun.on("data", messages(({ id, method }, bytes) => {
      if (method?.startsWith("Console.")) (console.frames++, (console.bytes += bytes));
      if (id === last) answered.resolve();
    }));
    upstream.pipe(bun);
    bun.pipe(upstream);
    bun.on("error", () => {});
    upstream.on("error", () => {});
  });
  await new Promise<void>(resolve => server.listen(0, "127.0.0.1", resolve));
  const proc = Bun.spawn({
    cmd: [process.execPath, "-e", `${prelude}; process.stdin.once("data", async () => { ${script}; process.stderr.write("\\n"); }); setInterval(() => {}, 1000);`],
    env: { ...process.env, BUN_INSPECT_CONNECT_TO: `tcp://127.0.0.1:${(server.address() as net.AddressInfo).port}` },
    stdin: "pipe",
    stdout: Bun.file("/dev/null"),
    stderr: "pipe",
  });
  await answered.promise;
  proc.stdin.write("go\n");
  proc.stdin.flush();
  let printed = "";
  for await (const chunk of proc.stderr) if ((printed += Buffer.from(chunk)).includes("\n")) break;
  // bun answers in order: when this request is answered, every frame of the script is at the relay.
  answered = Promise.withResolvers<void>();
  const body = JSON.stringify({ id: (last = 1e6), method: "Runtime.evaluate", params: { expression: "0" } });
  const head = Buffer.alloc(4);
  head.writeUInt32BE(body.length);
  socket.write(Buffer.concat([head, Buffer.from(body)]));
  await answered.promise;
  proc.kill();
  server.close();
  return { requests: requests.join(", "), console, printed: printed.trim() };
}

test("what the diagnostics socket costs a connected bun", async () => {
  const lines = await run(`for (let i = 0; i < 100000; i++) console.log(String(i).padStart(5, "0") + Buffer.alloc(80, "x"));`);
  console.log("requests:", lines.requests);
  console.log("100,000 console.log lines:", lines.console.frames, "Console.* frames,", lines.console.bytes, "bytes");
  const kept = await run(`
    for (let i = 0; i < 20000; i++) console.log({ request: i, body: new Uint8Array(10240) });
    Bun.gc(true); await Bun.sleep(100); Bun.gc(true);
    const stats = require("bun:jsc").heapStats();
    process.stderr.write(stats.objectTypeCounts.Uint8Array + " live Uint8Array, " + Math.round(stats.heapSize / 1e6) + " MB heap");`);
  console.log("after 20,000 logged 10 KB buffers and two GCs:", kept.printed);
  const transpile = `require(${JSON.stringify(join(import.meta.dir, "node_modules", "typescript"))}).transpileModule("let a: number = 1;\\n".repeat(2000), {})`;
  const warm = await run(
    `const cpu = process.cpuUsage();
    for (let i = 0; i < 20; i++) ${transpile};
    process.stderr.write(Math.round(process.cpuUsage(cpu).user / 1000) + " ms of CPU");`,
    `for (let i = 0; i < 40; i++) ${transpile}`,
  );
  console.log("20 TypeScript transpiles after 40 that ran before the requests:", warm.printed);
}, 120_000);